Condition Data Extraction Notes
The purpose of this document is to track Jake's findings when trying to run LGBM notebooks with new data from DNV.
Databases
As a summary, condition data came from prior databases:
- CCTV_Details.dbf
- CCTV_Headers.dbf
However, the latest database is an accdb format. I will be looking at the dbf databases to see if they are comparable to the newest accdb, then I can try to generate the HEADERS and DETAILS for the new dataset. The goal of this will be to export the new data in the same format as the old CSVs so I can easily merge the two CSVs together.
All new data extraction will be done in Python scripts and I will attempt to make them as generic as possible. Future video processing should be a much more streamlined process.
::important Anything related to LACP is irrelevant. These are related to laterals.
CCTV_Details.dbf
This database has only one table "CCTV_Details". It has the following columns:
- OBJECTID
- Header_ID
- FILENAME
- VIDEONUMBE
- PHOTOGRAPH
- DISTANCE
- CONTINOUSD
- CODE
- DIAMETER_D
- CLOCKFROM
- CLOCKTO
- PERCENTAGE
- INTRUSION
- JOBNUMBER
- REMARKS
- Type
- CodeID
- GlobalID
Normally, I would include an ERD here. Since there's only one table, however, this would be pointless.
CCTV_Headers.dbf
This database has only one table "CCTV_Headers". It has the following columns:
- OBJECTID
- ASSET_ID
- HEADER_ID
- FILENAME
- SURVEYEDBY
- CONTRACTNU
- JOBNUMBER
- CATCHMENT
- DIVISION
- DISTRICT
- PIPELENGTH
- Date
- Time
- LOCATION
- STARTMANHO
- SDEPTH
- SCOVER
- SINVERT
- FINISHMANH
- FDEPTH
- FCOVER
- FINVERT
- USESEWER
- DIRECTION
- SIZE1
- SIZE2
- SHAPE_1
- MATERIAL
- LINING
- PIPELENG_1
- TOTALLENGT
- YEARLAID
- VIDEONUMBE
- COMMENTS
- PURPOSE
- SEWERCATEG
- PRECLEANIN
- WEATHER
- LOCATIONCO
- FURTHERDET
- CUSTOMERNA
- GROUNDCOND
- CREW
- VIDEOFILEN
- STRUCTURAL
- STRUCTUR_1
- STRUCTUR_2
- SERVICE_TO
- SERVICE_ME
- SERVICE__1
- SERVICE_PE
- SERVICE__2
- SERVICE_GR
- NUMBEROFDE
- SPG
- HAS_ROOTS
- HAS_GREASE
- HAS_DEBRIS
- HAS_MSD
- HAS_MJD
- HAS_INT_SE
- HAS_GENERA
- HAS_INFILT
- HAS_OBSTRU
- INACTIVE
- EXTENSION
- GlobalID
Normally, I would include an ERD here. Since there's only one table, however, this would be pointless.
cameratruck database.accdb
